Nearly Optimal Control Scheme for Discrete-Time Nonlinear Systems with Finite Approximation Errors Using Generalized Value Iteration Algorithm

In this paper, a new generalized value iteration algorithm is developed to solve infinite horizon optimal control problems for discrete-time nonlinear systems. The idea is to use iterative adaptive dynamic programming (ADP) to obtain the iterative control law which makes the iterative performance index function reach the optimum. The generalized value iteration algorithm permits an arbitrary positive semi-definite function to initialize it, which overcomes the disadvantage of traditional value iteration algorithms. When the iterative control law and iterative performance index function in each iteration cannot be accurately obtained, a new design method of the convergence criterion for the generalized value iteration algorithm with finite approximation errors is established to make the iterative performance index functions converge to a finite neighborhood of the lowest bound of all performance index functions. Simulation results are given to illustrate the performance of the developed algorithm.